Skip to content

lab8 monitoring and alerting#1214

Open
giselesikeh wants to merge 5 commits into
inno-devops-labs:mainfrom
giselesikeh:feature/lab8
Open

lab8 monitoring and alerting#1214
giselesikeh wants to merge 5 commits into
inno-devops-labs:mainfrom
giselesikeh:feature/lab8

Conversation

@giselesikeh

Copy link
Copy Markdown

Goal

Add Lab 8 monitoring for QuickNotes using Prometheus, Grafana, one high-error-rate alert, a runbook, and bonus external synthetic monitoring.

Changes

  • Added Prometheus config to scrape QuickNotes at quicknotes:8080/metrics
  • Added Grafana datasource provisioning for Prometheus
  • Added Grafana dashboard provisioning with a QuickNotes Golden Signals dashboard
  • Added four dashboard panels: latency proxy, traffic, errors, and saturation
  • Added Prometheus alert rule QuickNotesHighErrorRate
  • Added runbook at docs/runbook/high-error-rate.md
  • Updated submissions/lab8.md with Task 1, Task 2, and bonus evidence
  • Reused the Lab 6 Dockerfile and healthcheck so QuickNotes can run as a healthy Compose service

Testing

  • Ran docker compose config
  • Ran docker compose up -d --build
  • Verified QuickNotes was healthy with docker compose ps
  • Verified Prometheus target was UP at http://localhost:9090/targets
  • Verified Grafana auto-loaded the QuickNotes Golden Signals dashboard
  • Generated traffic and confirmed dashboard graphs updated
  • Triggered malformed POST /notes requests for more than 5 minutes
  • Confirmed QuickNotesHighErrorRate moved to FIRING
  • Verified Checkly synthetic monitoring through ngrok with 100% availability, 316 ms average latency, and 529 ms p95 latency

Checklist

  • Title is a clear sentence (≤ 70 chars)
  • Commits are signed (git log --show-signature)
  • submissions/lab8.md updated

Signed-off-by: giselesikeh <giselesikeh17@gmail.com>
Signed-off-by: giselesikeh <giselesikeh17@gmail.com>
Signed-off-by: giselesikeh <giselesikeh17@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant